Existing agents for solving tasks such as ML engineering rely on prompting powerful language models. As a result, these agents do not improve with more experience. In this paper, we show that agents backed by weaker models that improve via reinforcement learning (RL) can outperform agents backed by much larger, but static models. We identify two major challenges with RL in this setting. First, actions can take a variable amount of time (e.g., executing code for different solutions), which leads to asynchronous policy gradient updates that favor faster but suboptimal solutions. To tackle variable-duration actions, we propose duration-aware gradient updates in a distributed asynchronous RL framework to amplify high-cost but high-reward actions. Second, using only test split performance as a reward provides limited feedback. A program that is nearly correct is treated the same as one that fails entirely. To address this, we propose environment instrumentation to offer partial credit, distinguishing almost-correct programs from those that fail early (e.g., during data loading). Environment instrumentation uses a separate static language model to insert print statement to an existing program to log the agent's experimental progress, from which partial credit can be extracted as reward signals for learning. Our experimental results on MLEBench suggest that performing gradient updates on a much smaller model (Qwen2.5-3B) trained with RL outperforms prompting a much larger model (Claude-3.5-Sonnet) with agent scaffolds, by an average of 22% across 12 Kaggle tasks.

Deep learning is widely used to uncover hidden patterns in large code corpora. To achieve this, constructing a format that captures the relevant characteristics and features of source code is essential. Graph-based representations have gained attention for their ability to model structural and semantic information. However, existing tools lack flexibility in constructing graphs across different programming languages, limiting their use. Additionally, the output of these tools often lacks interoperability and results in excessively large graphs, making graph-based neural networks training slower and less scalable. We introduce CONCORD, a domain-specific language to build customizable graph representations. It implements reduction heuristics to reduce graphs' size complexity. We demonstrate its effectiveness in code smell detection as an illustrative use case and show that: first, CONCORD can produce code representations automatically per the specified configuration, and second, our heuristics can achieve comparable performance with significantly reduced size. CONCORD will help researchers a) create and experiment with customizable graph-based code representations for different software engineering tasks involving DL, b) reduce the engineering work to generate graph representations, c) address the issue of scalability in GNN models, and d) enhance the reproducibility of experiments in research through a standardized approach to code representation and analysis.

Large language models (LLMs) have made significant progress in code generation tasks, but their performance in tackling programming problems with complex data structures and algorithms remains suboptimal. To address this issue, we propose an in-context learning approach that guides LLMs to debug by using a "print debugging" method, which involves inserting print statements to trace and analysing logs for fixing the bug. We collect a Leetcode problem dataset and evaluate our method using the Leetcode online judging system. Experiments with GPT-4 demonstrate the effectiveness of our approach, outperforming rubber duck debugging in easy and medium-level Leetcode problems by 1.5% and 17.9%.

In part 1, we learned how to use terraform to set up and manage our infrastructure conveniently. In this part, we will continue on our journey to deploy a running Stable Diffusion model on the provisioned cluster. Note: You can follow this tutorial end-to-end even if you're a free user (as long as you have some of free tier credits left). Let's take a look at what the final result would be. If you add a bit of noise to an image gradually for many steps, you will end up with an image containing noise.

From sifting through Twitter data to making your own Minecraft modifications, Python is one of the most versatile programming languages at a coder's disposal. The open-source, object-oriented language is also quickly becoming one of the most-used languages in data science. According to the Association for Computing Machinery, Python is now the most popular introductory language at universities in the United States. To help readers practice the Python fundamentals, datascience@berkeley gathered six coding problems, including some from the W200: Introduction to Data Science Programming course. Consider the following questions to make sure you have the proper prior knowledge and coding environment to continue.

In this article, we will discuss error handling in python with a try, except and finally keywords to handle file and data management. Exception handling is a very realistic approach to handle the exception when something wrong happens in the program, in that case, the interpreter stops and raises an exception until it is handled. To handle these types of exceptions in the program for continuing the execution we learn exception handling. Here, we get the output "2" because the print statement is executed. We can also print another statement to check the whole program is executed properly.

While this may be fine for notebooks targeted at teaching people about the machine learning process, in real projects it's a recipe for unmaintainable mess. The lack of good coding habits makes code hard to understand and consequently, modifying code becomes painful and error-prone. This makes it increasingly difficult for data scientists and developers to evolve their ML solutions. In this article, we'll share techniques for identifying bad habits that add to complexity in code as well as habits that can help us partition complexity.

Part 1 discussed the traditional machine learning (ML) pipeline and highlighted that manual feature extraction is not the right choice for working with large datasets. On the other hand, deep learning (DL) able to automatically extract features from such large datasets. Part 1 also introduced transfer learning to highlight its benefits for making it possible to use DL for small datasets by transferring the learning of a pre-trained model. In this tutorial, which is Part 2 of the series, we will start the first practical side of the project. This is by starting working with creating a Jupyter notebook and making sure everything is up and running. After that, the Fruits360 dataset is downloaded using Keras within the Jupyter notebook. After making sure the dataset is downloaded successfully, its training and test images are read into NumPy arrays which will be fed later to MobileNet for extracting features. This series uses the Jyputer notebook for transfer learning of the pre-trained MobileNet.

Before we jump into coding, let's understand the problem we are trying to solve. As humans, it is very easy for us to read and recognize a bunch of handwritten digits. Since this recognition is done unconsciously, we don't realize how difficult this problem actually is. Now imagine teaching a computer how to recognize these digits and writing out a set of rules (otherwise known as an algorithm) to tell the computer how to distinguish each digit from another. This proves to be quite a difficult task! Neural networks approach the problem of digit recognition very differently.